Dual sampling neural network: Learning without explicit optimization

脳型人工知能の実現に向けた新理論の構築に成功 --ヒントは脳のシナプスの「揺らぎ」--. 京都大学プレスリリース. 2022-10-24.Artificial intelligence using neural networks has achieved remarkable success. However, optimization procedures of the learning algorithms require global and synchronous operations of variables, making it difficult to realize neuromorphic hardware, a promising candidate of low-cost and energy-efficient artificial intelligence. The optimization of learning algorithms also fails to explain the recently observed criticality of the brain. Cortical neurons show a critical power law implying the best balance between expressivity and robustness of the neural code. However, the optimization gives less robust codes without the criticality. To solve these two problems simultaneously, we propose a model neural network, dual sampling neural network, in which both neurons and synapses are commonly represented as a probabilistic bit like in the brain. The network can learn external signals without explicit optimization and stably retain memories while all entities are stochastic because seemingly optimized macroscopic behavior emerges from the microscopic stochasticity. The model reproduces various experimental results, including the critical power law. Providing a conceptual framework for computation by microscopic stochasticity without macroscopic optimization, the model will be a fundamental tool for developing scalable neuromorphic devices and revealing neural computation and learning

Dynamics of Limit Cycle Oscillator Subject to General Noise

The phase description is a powerful tool for analyzing noisy limit cycle oscillators. The method, however, has found only limited applications so far, because the present theory is applicable only to the Gaussian noise while noise in the real world often has non-Gaussian statistics. Here, we provide the phase reduction for limit cycle oscillators subject to general, colored and non-Gaussian, noise including heavy-tailed noise. We derive quantifiers like mean frequency, diffusion constant, and the Lyapunov exponent to confirm consistency of the result. Applying our results, we additionally study a resonance between the phase and noise.Comment: main paper: 4 pages, 2 figure; auxiliary material: 5-7 pages of the document, 1 figur

Synchronization of Excitatory Neurons with Strongly Heterogeneous Phase Responses

In many real-world oscillator systems, the phase response curves are highly heterogeneous. However, dynamics of heterogeneous oscillator networks has not been seriously addressed. We propose a theoretical framework to analyze such a system by dealing explicitly with the heterogeneous phase response curves. We develop a novel method to solve the self-consistent equations for order parameters by using formal complex-valued phase variables, and apply our theory to networks of in vitro cortical neurons. We find a novel state transition that is not observed in previous oscillator network models.Comment: 4 pages, 3 figure

Effective long-time phase dynamics of limit-cycle oscillators driven by weak colored noise

An effective white-noise Langevin equation is derived that describes long-time phase dynamics of a limit-cycle oscillator subjected to weak stationary colored noise. Effective drift and diffusion coefficients are given in terms of the phase sensitivity of the oscillator and the correlation function of the noise, and are explicitly calculated for oscillators with sinusoidal phase sensitivity functions driven by two typical colored Gaussian processes. The results are verified by numerical simulations using several types of stochastic or chaotic noise. The drift and diffusion coefficients of oscillators driven by chaotic noise exhibit anomalous dependence on the oscillator frequency, reflecting the peculiar power spectrum of the chaotic noise.Comment: 16 pages, 6 figure

Robustness of the noise-induced phase synchronization in a general class of limit cycle oscillators

We show that a wide class of uncoupled limit cycle oscillators can be in-phase synchronized by common weak additive noise. An expression of the Lyapunov exponent is analytically derived to study the stability of the noise-driven synchronizing state. The result shows that such a synchronization can be achieved in a broad class of oscillators with little constraint on their intrinsic property. On the other hand, the leaky integrate-and-fire neuron oscillators do not belong to this class, generating intermittent phase slips according to a power low distribution of their intervals.Comment: 10 pages, 3 figure